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Abstract 

We introduce new techniques for proving lower bounds on the running time of randomized 
algorithms for asynchronous agreement against powerful adversaries. In particular, we define 
a strongly adaptive adversary that is computationally unbounded and has a limited ability 
to corrupt a dynamic subset of processors by erasing their memories. We demonstrate that 
the randomized agreement algorithms designed by Ben-Or and Bracha to tolerate crash or 
Byzantine failures in the asynchronous setting extend to defeat a strongly adaptive adversary. 
These algorithms have essentially perfect correctness and termination, but at the expense of 
exponential running time. In the case of the strongly adaptive adversary, we show that this 
dismally slow running time is inherent: we prove that any algorithm with essentially perfect 
correctness and termination against the strongly adaptive adversary must have exponential 
running time. We additionally interpret this result as yielding an enhanced understanding 
of the tools needed to simultaneously achieving perfect correctness and termination as well 
as fast running time for randomized algorithms tolerating crash or Byzantine failures. 

1 Introduction 

Achieving agreement in a distributed system despite failures is a central problem in distributed 
computing. We consider a complete network of n processors able to communicate with each 
other by passing messages. Initially, each processor has an input bit. The task is to design a 
failure-resilient protocol that allows all non-faulty processors to agree on an output value, with 
the restriction that it must be equal to at least one of their inputs (this rules out the trivial 
solution of having a constant decision value independent of the inputs). The difficulty of this 
problem depends heavily on several additional specifications that must be made. In particular, 
is communication synchronous or asynchronous? What kinds of failures should be tolerated? 
If the errors and/or scheduling are controlled by an adversary, what resources and information 
does the adversary have access to? 

We will consider a very challenging setting of asynchronous communication where message 
scheduling is controlled by an adversary with unbounded computational power who is given 
unrestricted access to all message contents and internal states of all processors. The adversary 
will also be empowered to cause limited types and quantities of processor failures. In this work, 
we will consider two kinds of failures: crash failures, which cause a processor to quit without 
warning, as well as resetting failures, which we will define and motivate below. 

In this setting, the elegant result of Fischer, Lynch, and Paterson [13] shows that it is already 
impossible to design a deterministic protocol for agreement that always terminates, even if the 
adversary is limited to causing at most one crash failure. A common approach for tolerating 
this obstacle in practice is to use an algorithm that terminates as long as worst-case scheduling 
does not occur indefinitely. This is a property achieved by the well-known Paxos algorithm 
constructed by Lamport |21| . Randomized algorithms provide a potential alternative. Quickly 
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following the impossibility result, Ben-Or [6J and Bracha [9] presented randomized algorithms 
terminating with probability one, even against such strong adversaries. These algorithms were 
intuitively structured, and Bracha's algorithm tolerated an optimal number of failures, namely 
allowing for t processors to behave in an arbitrary malicious fashion, for any t < ^. However, 
for some settings of the initial input bits, the algorithms of [6j|9] run for time that is exponential 
in n (with high probability) when t = O(n). 

The algorithms in [61 19] seem to provide even stronger failure resilience than is captured by 
the adversarial model employed. In particular, the proofs of correctness rely only on the fact 
that at most t processors are faulty at one time, where the notion of "time" must be defined 
in an appropriate (and perhaps subtle) way. This gives some hope for recovering from even 
more than t < ^ failures over the course of long executions if individual processor faults are 
fleeting occurrences. In particular, one might suppose that faulty processors could be detected 
and fixed during the course of a protocol execution, thereby allowing for more total failures. 

In order to more fully characterize the failure resilience provided by the basic algorithm 
underlying [6, [9], we define the notion of resetting failures. A resetting failure at a processor 
results in loss of internal state: a processor that is reset is assumed to lose the entire contents 
of its memory (except for its initial input bit and its output bit). A resetting failure can model 
a processor that is detected to be faulty and has its memory reset in order for it to rejoin the 
protocol as a non-faulty processor. We define a strongly adaptive adversary who can cause up 
to t resetting failures in a certain window of time, where our measure of time is appropriately 
linked to the events of the execution. (Some kind of linking is necessary to avoid allowing the 
adversary to always cause a failure at the processor currently taking a step in the execution, 
for example.) 

We prove that a simple variant of the algorithms in [61 [9] is indeed successful against such 
a strongly adaptive adversary (with probability one). Of course, this retains the exponentially 
slow running time. We then show that exponential slowness for t = f2(n) is inherent to any 
algorithm achieving success with probability one against this strongly adaptive adversary. This 
provides a rather complete understanding of what is achievable in the presence of adaptive 
resetting faults. 

In contrast, the relatively recent algorithm of Kapron et. al. [15] runs very quickly (poly- 
logarithmic time in n) and tolerates t < — e) non-adaptive Byzantine failures, but incurs a 
non-zero probability of non-termination or termination with invalid outputs. It is an interesting 
question to study to what extent the sacrifices made here (non-adaptivity, non-zero probability 
of incorrect output) are necessary to achieve fast running time. The algorithm in [T5] works 
by iteratively dividing the processors into small "committees" that can afford to run the slow 
algorithm of [9j to hold elections to select random smaller subsets of processors to continue into 
new committees. A single final committee is reached that, with 1 — o(l) probability, contains 
a suitably bounded percentage of faulty processors. This final committee runs the algorithm of 
[9] and informs the other processors of the result. 

It is clear that this approach cannot be used against an adaptive adversary, who can simply 
wait for the final committee to be determined and then cause faults. This approach also seems 
to inherently incur non-zero probability of an invalid result, as there is always a nonzero chance 
that the final committee is composed entirely of faulty processors. With the goal of beginning 
a systematic study of what can be achieved without incurring these disadvantages, Lewko [23] 
previously proved that a class of algorithms generalizing Ben-Or and Bracha's algorithms in 
[6j[9] cannot achieve sub exponential running time against an adversary causing t = Q(n) non- 
adaptive Byzantine failures. The class of algorithms was restricted in several ways, including 
a constant bound on the support size of all message distributions sampled by processors and a 
requirement for received messages from different processors to be treated symmetrically. (For 
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a more detailed description of the algorithm class, see [25].) 

The techniques we introduce to prove the lower bound against strongly adaptive adversaries 
can be applied in this setting to yield an exponential lower bound on running time for a new 
class of algorithms tolerating t = Cl(n) crash failures. This class is incomparable to the class 
considered in [23], and this result yields several new insights. Most notably, our lower bound 
technique can tolerate arbitrary use of randomness by the processors, allowing us to avoid 
requiring any restriction on the support size as in [23J. We also avoid any requirement of 
symmetry in how received messages are treated, and our class is more intuitively defined. 

Concurrently with this work, King and Saia [18J have discovered a Las Vegas polynomial- 
time algorithm tolerating adaptive Byzantine faults that falls outside the classes of algorithms 
considered here and in [23]. This implies a separation between what can achieved against the 
classical adaptive Byzantine adversary and the strongly adaptive adversary. 

Our Techniques To prove the exponential lower bound on running time, we rely crucially 
on a general probabilistic inequality of Talagrand |27| . which roughly states that any product 
distribution J7i x x . . . x Q n cannot put too much weight simultaneously on two sets A and B 
in n-dimensional space that are "far apart." For our purposes, "far apart" can be interpreted as 
having Hamming distance Q.(n). We also can interpolate this result: if some product distribution 
ill x ... x puts significant weight on A and some other product distribution 111 x . . . x U n 
puts significant weight on B, then there is some mixed product distribution fii x ... x Qi x 
n.; + i x ... x n„ that puts small weight on each of A and B. 

To use these tools in order to prove a lower bound on running time, we define iterative pairs 
of sets in the joint state space of the n processors that represent different levels of progress 
towards a final decision. By leveraging the capabilities of the strongly adaptive adversary (or 
later by leveraging the defining properties of the algorithm class), we prove that each of these 
pairs of sets is sufficiently separated in Hamming distance. We then apply the probabilistic 
inequalities repeatedly as an execution travels through the state space of the n processors, 
showing that for some initial setting of the inputs, the adversary can prevent the algorithm 
from making much progress in a given window of time with high probability. This ultimately 
yields our lower bound. 

Our approach of leveraging general properties of product distributions represents a mean- 
ingful expansion of the suite of available tools for proving lower bounds in a distributed setting. 
In particular, there are essentially only a few core tools for proving lower bounds for randomized 
algorithms, and previous approaches do not achieve exponential lower bounds on running time 
when arbitrary amounts of randomness can be used. We consider our new techniques to be 
the main contribution of this work. In the following subsection, we briefly survey prior lower 
bounds and other relevant work. 

1.1 Related Work 

The problem of reaching agreement despite faults was introduced by Pease, Shostak, and Lam- 
port in [25], who also proposed the Byzantine failure model in [22] , Since its introduction, the 
problem of fault-tolerant agreement has been widely studied in a variety of models. Several 
works have considered computationally bounded adversaries, a setting in which cryptographic 
tools can be employed ([261 123 El ED Ell 121]) for example). In the synchronous communi- 
cation setting, polylogarithmic round randomized protocols for Byzantine agreement against 
non-adaptive adversaries were obtained in [191 [2Q1 El El] ■ Recent work has focused on reducing 
the communication overhead of synchronous protocols [16} H7] . 

Several lower bounds are also known. In addition to the impossibility of deterministic 
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algorithms in the asynchronous setting mentioned above, there is a sharp lower bound of t 
rounds for deterministic algorithms in the synchronous setting [12] . This lower bound is proven 
by assembling a chain of executions where any two adjacent executions are indistinguishable to 
some non- faulty processor and the two ends of the chain represent different decision values. This 
basic strategy is adapted and expanded in [23] to yield a lower bound for a class of randomized 
algorithms, but this class inherently limits the amount of randomness used in choosing an 
individual message. 

Polynomial lower bounds for randomized algorithms include the result of Bar-Joseph and 
Ben-Or [5], which proves a lower bound of t/y/n logn on the number of expected rounds for 
a randomized synchronous protocol against an adversary who can adaptively choose to fail 
t processors. Another lower bound is due to Attiya and Censor [3], who show that for any 
integer k, the probability that a randomized Byzantine agreement algorithm does not terminate 
in k(n — t) steps is at least l/c k for some constant c. Aspnes [2j proves a lower bound of 
Q(i/log 2 i) on the expected number of local coin flips for asynchronous algorithms against 
adaptive adversaries that holds in either the shared memory or message passing model. This 
result is proven by establishing an extension of the techniques in [13J to a randomized setting. 
In the shared memory model, there are polynomial time randomized algorithms tolerating crash 
failures, and tight bounds on their total step complexity are proven by Attiya and Censor in 

a- 

2 Models and Definitions 

We let n denote the total number of processors, and consider each processor to be endowed 
with a unique identity between 1 and n. We let < t < n be a fixed positive integer (we 
let t be arbitrary for the purposes of definition, but note that in our theorems below, we will 
take t = cn for a suitably small positive constant c). We assume that each processor has 
its own source of random bits, and all of these sources are unbiased and independent. Each 
processor also has a fixed input bit, and a write-once output bit that is initially set to JL. We 
work in a message-passing model, where any single processor can send a message to any other 
processor along a dedicated "message channel," meaning that the recipient of a message will 
always correctly identify the sender. We let Ai denote the space of all possible messages (this 
can be infinite). An element m G Ai contains a sender identity, a receiver identity, and a string 
of bits interpreted as its contents. 

We define the state of a processor to include the current contents of its memory (note that 
this holds its identity, its input bit, and its output bit with current value 0, 1, or _L). We let £ 
denote the set of possible processor states. An n-tuple of states, a £ T, n specifies a configuration 
for the n processors. 

An algorithm A is a collection of probability distributions on E x Ai n , parameterized by 
SxAi. In other words, an algorithm specifies how a processor should sample a new state and 
outgoing messages, depending on the current state as well as a just received message. The new 
state may contain updated memory contents (the output bit may or may not change). The new 
sent messages can depend on the freshly received message, the current memory of the processor, 
and freshly sampled random bits. We include G Ai to allow a processor to choose not to send 
a message. 

We will adopt the usual notion of asynchrony and imagine that message delivery is con- 
trolled by an adversary. We will allow our adversary complete access to the current states of 
all the processors and the contents of all messages. We also allow our adversary unbounded 
computational power. 

It is typical to define an execution as a sequence of steps, where each step consists of 
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a processor (potentially) receiving a message, performing some local computation, and then 
possibly placing some outgoing messages into a "message buffer." The adversary then controls 
the sequence of steps by deciding which processor will take the next step and what message (if 
any) that processor will receive. 

To model an adversary able to crash up to t processors, one can insist that in any infinite 
execution, all but at most t processors take infinitely many steps and that every message sent 
to an infinitely stepping processor is eventually delivered. It is also common to consider a 
stronger Byzantine adversary, who instead has the power to corrupt the messages sent by up to 
t processors. In this setting, we may require all processors to take infinitely many steps - but 
note that corrupted processors may simulate crashed processors by maliciously choosing not to 
send messages (changing a non-empty message m to is considered a permissible corruption 
by the adversary). We note that corruption of messages allows an adversary to lie about the 
random coins sampled by a limited number of processors. 

We will instead define a adversary who is able to "reset" a changing set of < t processors. 
Resetting a processor will correspond to erasing the contents of its memory, except for its input 
bit, its output bit, its processor identity, and a special counter that will increment each time a 
reset occurs. We assume that a processor keeps a local copy of the counter's value in its state, 
and hence will detect a reset when the local copy is erased and the real counter is non-zero. This 
mechanism of detection is just a book-keeping device, the key point is that we are assuming 
resets are events processors can internally detect (note that this strengthens our lower bound 
result). 

We now consider executions expressed as sequences of more fine-grained steps between con- 
figurations, where we allow three distinct types of steps. A resetting step will cause the memory 
of a specified processor to be reset. A receiving step will deliver a message from the message 
buffer to its intended recipient. The recipient will then perform a local computation (perhaps 
sampling from some fresh local randomness). This will be the only kind of step that involves 
randomization. 

Finally, a sending step will allow a processor to place a set of new messages into the message 
buffer (this set may be empty if the processor chooses not to send anything). We assume 
that a single sending step represents a complete response to prior events, meaning that if a 
processor takes a sending step and then takes another sending step without taking any resetting 
or receiving steps in between, then the second sending step will have no effect - the state of the 
processor will remain unchanged and no new messages will be sent. The adversary will control 
the order and nature of the steps. 

Given a partial execution expressed as a finite sequence of such steps, we define its probability 
(with respect to a fixed algorithm A) to be the product of the probabilities of each state change 
induced by a step, under the distributions specified by the algorithm. (This is assuming the 
initial configuration is valid.) Note that this will be zero if deterministic transitions are not 
followed, or if a step indicates delivery of a message that was never sent, etc. 

Naturally, it would be impossible to make progress against an adversary allowed to reset 
processors arbitrarily. In particular, we must design a model that rules out the trivial case of an 
adversary that resets the memory of the receiving processor after every message delivery, as no 
algorithm can make progress under such adverse circumstances. To ensure feasibility, we could 
limit the adversary to resetting at most t processors throughout an execution. However, we can 
achieve success against even stronger adversaries. To specify an interesting such adversary, we 
make one key definition: 

Definition 1. An acceptable window is a consecutive segment of steps of the following form. 
First, all n processors take sending steps. Then, for sets S\, S2, ■ ■ ■ , S n C [n] all of size > n — t, 
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a sequence of receiving steps follows that delivers to each processor i the messages just sent to 
it from processors in the set Si. Finally, a sequence of at most t resetting steps occurs. 

The notion of an acceptable window is a formal unit of "time" during an execution in which 
at most t processors are faulty, and hence the other processors may not receive any messages 
from them. One could imagine adding a requirement that i G Si for each i, as a processor can 
always safely wait to receive a message from itself, but this is unnecessary, as no resets occur 
between sending and receiving. This means that any information the processor could pass to 
itself through a message can instead be stored directly in the processor's state. Thus, adding a 
requirement that i G Si would be superfluous here. 

We define the Strongly Adaptive Adversary to be an adversary allowed to reset processors 
and control message sending and receiving up to the constraint that any infinite execution is 
composed entirely of adjacent, disjoint acceptable windows. We observe that this adversary is 
incomparable to the usual Byzantine asynchronous adversary. Our strongly adaptive adversary 
has the additional power to erase processor memory, but it lacks the power to have corrupted 
processors "lie" about their local random bits. 

Our use of the phrase "the strongly adaptive adversary" is a bit imprecise, since this tech- 
nically constitutes a class of adversaries in the following sense. A single adversary should be 
thought of as a deterministic function that maps a partial execution to a next applicable step. 
Such a function need not be efficiently computable. We will consider the class of strongly adap- 
tive adversaries to be the collection of individual adversaries that satisfy the requirement above 
to produce acceptable windows. 

It may seem a bit unnatural to impose the constraint that an adversary should stick to 
acceptable windows, but we feel this captures the intuitive notion that the adversary should only 
corrupt t processors "at one time," as otherwise progress would be impossible. Furthermore, as 
our main result in this model is a lower bound, placing restrictions on the adversary strengthens 
our result. 

We call a configuration reachable if it occurs as the consequence of some partial execution 
with non-zero probability that is decomposable as a concatenation of acceptable windows. Note 
that the notion of reachability depends on the algorithm employed. 

Definition 2. We say an algorithm A achieves measure one correctness against all strongly 
adaptive adversaries if any reachable configuration contains only agreeing or _L output bits, ( in 
other words, one output bit being and another being 1 is disallowed, but any assortment of O's 
and _L 's or any assortment of l's and ±'s is allowed). We also require that when an output is 
not _L, it must agree with one of the inputs. This means that if all processors have inputs equal 
to 0, the decision cannot be 1, and vice versa. 

Definition 3. We say an algorithm A achieves measure one termination if any infinite 
execution (composed of acceptable windows) in which some processor taking an infinite sequence 
of sending and receiving steps never sets its decision bit has probability zero (we define the prob- 
ability of an infinite execution to be the limit of the probabilities of its finite partial executions). 

In an asynchronous setting, defining the running time of an execution can be a bit subtle. 
One typical measure is to consider the length of the longest message chain before a decision is 
reached, where a message chain includes messages mi, m2, ■ ■ ■ , mt such that mi is received by 
the sender of mj+i, at some point prior to the sending of mj+i. It is not immediately clear how a 
"message chain" should be defined in the presence of resetting faults: should a message sent after 
a reset be counted as continuing a chain of messages received before the reset? Since our strongly 
adaptive adversary is constrained to keep to schedules that are approximately synchronous, we 
will employ a more obvious measure, namely the number of acceptable windows that pass before 
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the first processor decides. When we later reformulate our techniques to obtain a lower bound 
for a class of algorithms in the presence of crash failures, we will define the running time of an 
execution as the length of the longest message chain preceding a decision. 

3 Feasibility Against the Strongly Adaptive Adversary 

Both Ben-Or |6j and Bracha [9j provide expected exponential time algorithms for Byzantine 
agreement against a full-information asynchronous adversary (terminating and succeeding with 
probability 1). Bracha's algorithm introduces a bit more complexity in order to achieve the 
optimal resilience of t < ^ in the Byzantine setting. 

Inspired by these algorithms, we provide a close variant that succeeds against the strongly 
adaptive adversary. We will not be concerned with obtaining the optimal resilience, and so 
will favor simplicity of presentation over possible improvements to the constant fraction of 
resets allowed per acceptable window. The algorithm is parameterized by several thresholds, 
T\ > T2 > T3 , and we will discuss appropriate settings of these below. 

Throughout the algorithm, each processor p will store its input bit, its (write-once) output 
bit, and a few additional variables. The variable r p will hold the current "round number" and 
is initialized to 1. The variable x p is initialized to be equal to the input bit of processor p. 

step 1: Send the message (r p ,x p ) to all processors. 

step 2: Wait until T\ messages of type (r q , x q ) have arrived from other processors with values 
of r q = r p . 

step 3: If at least T2 of these T\ messages have the same bit value v for the last entry, then 
write v to the output bit (assuming this bit is not yet written). If at least T3 of these T\ 
messages have the same bit v, then set x p = v. Otherwise, set x p to be a freshly sampled 
uniformly random bit. 

step 4: Set r p = r p + 1 and return to step 1. 

handling resets To address resets, a processor p also does the following. If p has just been 
reset (an event that is detectable), then processor p waits to receive at least T\ messages (r q , x q ) 
from other processors with a common value of r. It then sets its own r p value to the match this 
and returns to step 3 above (note that a newly reset processor refrains from sending messages 
until it resumes normal operation). 

We note that 2T3 > T\ should hold in order for the behavior in step 3 to be clear. This is a 
constraint we will always impose. 

Theorem 4. The above algorithm achieves measure one correctness and termination against 
the strongly adaptive adversary for t < ^ when the thresholds T±,T2, T3 are set to satisfy n — 2t> 
T 1 >T 2 >T 3 + t, and 2T 3 > n. 

Proof. We note that whenever t < ^, the constraints on the thresholds T%,T2, T3 specified above 
are achievable by setting T\ := n — 2t, T2 = T%, T3 = n — 3t. (Having a smaller value of t allows 
one to set T2 smaller than T\ , which will lead to improvement in running time but is not relevant 
for measure one correctness and termination.) 

We first establish measure one correctness. Suppose that a is a reachable configuration 
in which some processor has decided. We consider a non-zero probability partial execution 
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composed of acceptable windows leading to a (such a partial execution must exist by definition 
of reachability). Now, we let w denote the earliest acceptable window in which a decision is 
made, and we let p denote a processor deciding in window w. In order to decide on a bit v, 
p must have received > T2 messages of the form (r, v) for its current value of r p . Each other 
processor q must have received > T2 — t of these messages (r, v) during the receiving steps in 
window w. 

We now consider how the internal round numbers r maintained by the processors evolve 
during a sequence of acceptable windows. Initially, all round numbers are 1. Assuming that 
n — t > Ti, all processors will increment r to be 2 during the first acceptable window. Thus, 
entering the second acceptable window, all processors that were not reset during the first window 
will have r = 2, and the reset processors will have blank r values (denoted by _L). During the 
second acceptable window, each processor will receive at least n — 2t messages from processors 
with r values equal to 2. Assuming that T\ <n — 2t, every processor will then have r = 3 before 
the resetting steps. 

Extending this reasoning inductively, we see that in window w, at least n — t processors will 
enter the window with r = w (with the rest having r = _L). Again assuming that T\ < n — 2t 
and additionally assuming that T2 — t > T3, every processor will have x q = v and r = w + 1 just 
before the resetting steps that conclude window w. It follows that every processor who has not 
yet decided will decide v in window w + 1. We must also check that it is impossible for some 
processor to decide the opposite of v during window w. This is impossible as long as 2T2 > n. 

We have thus shown that it is impossible to obtain contradicting decision values in a reach- 
able configuration. To see that decision values conflicting with a unanimous setting of the inputs 
are also impossible, note that if all inputs are equal to a common value v, then all processors will 
decide v in the first acceptable window. This completes our proof of measure one correctness. 

To establish measure one termination, we first argue that during any given acceptable win- 
dow, no two processors p and q can fix x p and x q deterministically to conflicting values. If p 
deterministically sets x p to v, this means it received > T3 messages with the value v. Processor 
q could not have received > T3 messages with the opposite value if we impose the constraint 
that 2T3 > n. Assuming this constraint, no two processors can deterministically set conflicting 
values. Thus, there is at least a 2~ n probability that all processors p set the same value for x p 
during any given window. Thus, the probability of not deciding approaches as the number of 
acceptable windows approaches infinity. This implies measure one termination. □ 

We observe that for any constant c < g, setting t = cn makes measure one correctness and 
termination attainable through the above algorithm, but the algorithm will incur exponential 
running time (with high probability) against an adversary that chooses initial inputs evenly 
split between and 1. To see this, note that T3 will always need to be > ^n, and hence T2 will 
always need to be > (| + c)n. Decision will then be contingent on obtaining a strong majority 
that occurs with probability that is exponentially small (depending on c). This is a consequence 
of the simple fact that with high probability, a sampling of n independent uniformly random 
bits will yield a deviation of only 0(n2 +e ) from the mean (for any small e > 0). Hence, with 
high probability per round, the adversary can continually extend the execution to last one 
more round without deciding by showing every processor an approximate split between and 
1 messages, and then having all of them set their next bits randomly in step 3. We expect this 
to continue for an exponential number of rounds until a strong enough majority happens by 
chance to prevent the adversary from continuing in this fashion. 
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4 Impossibility of Expected Polynomial Time Against the Strongly 
Adaptive Adversary 



Here we establish our main result: an exponential lower bound on the running time for any 
algorithm achieving measure one correctness and termination against the strongly adaptive 
adversary. 

Theorem 5. We sett = cn where c > is any fixed positive constant. Then there exist absolute 
positive constants C,a ( depending only on c) such that, for any algorithm achieving measure 
one correctness and termination, there is a strongly adaptive adversary and a setting of the 
inputs bits such that with probability > \, the running time is > Ce an . 

We first give a high-level outline of our proof. As a base case, we observe that the reachable 
configurations corresponding to a decision of and the reachable configurations corresponding 
to a decision of 1 form two sets (denoted Zq and Z®) that are significantly separated in Hamming 
distance. Intuitively, if conflicting decision states were too close, then the differing processors 
could be temporarily silenced, and the other processors could be forced to make a decision that 
could conflict. By interpolating over the input possibilities and applying Talagrand's inequality, 
we could use this base observation to prove that there is a setting of the inputs such that 
reaching a decision in just one "round" of communication is very unlikely. 

To work up to analyzing many rounds, we inductively build pairs of sets Zq and Z\ of 
configurations further out from decisions. These pairs of sets will remain Hamming-separated 
and will be designed so that if a configuration is not in Zq, say, then the adversary will have a 
good chance of continuing the execution for k more rounds without a decision of occurring. We 
define Zq such that if we start from a configuration in Zq and apply certain acceptable windows, 
then there is always a sufficient chance of landing in Zq~ 1 . We define Z\ analogously. To show 
that Z^ and Z\ are still significantly separated in Hamming distance, we argue that if they 
were too close, this would imply the existence of a single product distribution placing too much 
weight simultaneously on Zq~ 1 and Z^~ l . Since these are assumed to be Hamming-separated 
by the inductive hypothesis, this would contradict Talagrand's inequality. 

We then show that as long as one avoids the union of the sets Zq and Z\, then there is an 
acceptable window that can be used to extend the execution to a state avoiding Zq~ 1 U 
with high probability. This is essentially an interpolation argument: since we know there is a 
choice of window that avoids Zq~ 1 and another that avoids Z\~ x , we can interpolate to obtain 
a single choice of extending window that avoids the union. Finally, we show that if one begins 
outside of Zq U Z\, then one can extend the execution for k steps without a decision occurring 
with constant probability for a value of k that is exponential in n. 

To prove this theorem formally, we develop some key lemmas and definitions in the next 
subsections. 

4.1 A Probabilistic Lemma 

We will crucially rely on Talagrand's inequality, a very general tool for studying product mea- 
sures. We will state a consequence of it here in the context of Hamming distance, as we will 
not need the additional generality provided by the full statement. A fuller statement and proof 
can be found in pQ, for example. 

We first develop some convenient notation. We let = Y\a=i where each is a prob- 
ability space and is endowed with the product measure. We employ the usual notion of 
Hamming distance between points in Q: for x = (x±, . . . ,x n ) and y = (y\, . . . ,y n ) 6 $7, we 
define A(x, y) to the number of coordinates i such that Xi / y; L . Given a set A C f2 and a point 
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x = (xi, . . . , x n ) G fi, we define the Hamming distance A(x, A) between the point x and the set 
A to be the minimal Hamming distance attained between x and a point a £ A: 

Definition 6. For A C Q and x 6 fi, 

A(x, A) := min{A(x, a) : a G A}. 

Similarly, we define the Hamming distance between two sets A, B C Q, to be the minimal 
Hamming distance attained between a point c£ i and a point b £ B: 

Definition 7. For i.BCfi, 

A(A, 5) := min{A(a, b) : a £ A,b E B}. 

Finally, given a set iCSl and a non-negative real number d, we define the set B(A, d) to 
be the subset of points in 0, which are at a Hamming distance of at most d from A: 

Definition 8. For A C Q and d > 0, 

B(A,d) :={x£Q: A(x,A) < d}. 

We are now prepared to state the required consequence of Talagrand's inequality (see [I], 
for example): 

Lemma 9. For any ACQ and any d > 0, 

F[A](l-F[B(A,d)]) < e~£. 
4.2 The Building Blocks of the Proof 

We now recursively define two sequences of subsets of S n that will form the building blocks of 
our proof of Theorem [5j These definitions will be made with respect to a fixed algorithm A 
and a threshold parameter r > that we will set later. Our base sets are defined as follows: 

Definition 10. We let Z® denote the set of reachable configurations in T, n such that at least 
one processor has written to its output bit. Similarly, we let Zf denote the set of reachable 
configurations in S n such that at least one processor has written 1 to its output bit. 

Lemma 11. If the algorithm A satisfies measure one correctness and measure one termination, 
then A (Zg, Zf) > t. 

Proof. We suppose not. Then there exist reachable configurations a, 7 G S n such that a G Z®, 
7 G Z®, and A(cr, 7) < t. Without loss of generality, we suppose that a and 7 only differ in the 
first t coordinates (i.e. only in the local states of processors 1 through t). Consider a non-zero 
probability partial execution composed of acceptable windows that results in configuration a. 
The adversary can continue such an execution by always delivering the messages from the last 
n — t processors. This will allow an arbitrarily long sequence of new acceptable windows. 

Since the algorithm A satisfies measure one termination, if the adversary keeps extending 
this execution by appending new acceptable windows, with probability one a decision must 
eventually be reached, and since a G Z®, this decision must be (with probability one). How- 
ever, we can apply the same argument to a partial execution reaching 7 and then similarly 
delivering messages only from the last n — t processors. Since the distribution of the states of 
the last n — t processors is the same in both cases, it must be that their decision is also the 
same. Since 7 G Zf, this contradicts measure one correctness. □ 
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Given sets R, Si, . . . , S n C [n] satisfying \R\ <t, \Si\ > n — t Vi, we say the strongly adaptive 
adversary can appZy this set to a reachable configuration er € X ra , meaning that the adversary 
can execute sending steps for all processors, deliver to each processor i the messages sent to 
it by senders in Si, in some fixed order, and then reset the processors in R. Note that the 
application of sets R, Si, . . . , S n with the specified properties results in an acceptable window 
(by definition). 

Once we have defined sets Zq~ 1 and Z^~ l for some positive integer k, we define the next 
sets Zq and Z\ as follows: 

Definition 12. We let Zq denote the set of reachable configurations in S n such that, for any 
sets R, S such that \R\ < t, \S\ > n—t, the adversary applying R, S, S, . . . , S to the configuration 
will result in a new configuration that belongs to Z^ 1 with probability > r. Similarly, we let 
Z\ denote the set of reachable configurations in S n such that, for such R and S, the adversary 
applying R, S, S, . . . , S to the configuration will result in a new configuration that belongs to 
Z\~ x with probability > r. 

_ii 

Lemma 13. If the algorithm A satisfies measure one correctness and termination and r > e s« , 
then A (Zq, Z\\ > t for all non-negative integers k. 

Proof. We proceed by induction on k. The base case k = is addressed above in Lemma [TT1 
We assume the result holds for k — 1 > 0, and we suppose it is false for k. Then there exist 
reachable configurations a, 7 E S n such that a E Zq, 7 E Z\ , and A (17,7) < t. Without loss of 
generality, we suppose that a and 7 only differ in the first t coordinates. We let R denote the 
set {1,2,..., t} and S denote the set {t + 1, t + 2, . . . , n}. By definition of Zq, if the adversary 
applies R, S, . . . , S to a, this will with probability > r result in a new configuration belonging 
to Zq^ 1 . Similarly, if the adversary applies R, S, . . . , S to 7, this will with probability > r result 
in a new configuration belonging to Z\~ x . 

We first suppose that both a and 7 are configurations in which no decisions have occurred. 
In other words, no processors have yet written to their output bits. Assuming this, the resets 
will obliterate the differences between the first t processor states. Hence the distribution of 
the resulting configuration is identical in these two cases, as it is independent of the prior 
contents of the memories of the reset processors, as these have been erased (and their messages 
went undelivered). Since local randomness is sampled independently by each processor only in 
response to the message receipts in the window, which occur after the deterministic sending 
steps, the distribution on the resulting configuration (which is reachable with probability one) 
is in fact a product distribution. This distribution places weight > r on each of two sets, Zq~ 1 
and Z\~ x , that are separated by a Hamming distance > t. Applying Lemma [U we thus have 
that 

.2 ,2 

:y t t 

r < e 4n 44> r < e sn . 

This contradicts our stipulation on the value of r. 

We finally consider the case where some decision has already been made in a. Since a E Zq , 
this decision must be 0. However, repeatedly applying acceptable windows of R, S, . . . , S to a 
must result in a decision of 1 with nonzero probability, since 7 E Z\, and the distribution of 
the final n — t states here is independent of the output bits of the first t processors, as their 
messages are never delivered. This contradicts measure one correctness. □ 

We now prove that if a reachable configuration is not in Zq or Z\, then the adversary can 
choose the next acceptable window in a way that will (with high probability) avoid landing 
in Zq~ 1 U Z\~ x . The intuition for this can be developed as follows. We know that there is a 
product distribution induced by an acceptable window that places low probability on Zq -1 , and 
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we know there is a (potentially different) product distribution induced by an acceptable window 
that places low probability on Z^~ l . We will obtain a single product distribution that places 
low probability on both sets simultaneously by interpolating between these two distributions. 
We use the fact that Lemma [9] yields graceful degradation in the quality of the threshold for 
"low probability" as we perturb one coordinate of the product distribution at a time. If we 
interpolate carefully, we can also ensure that the interpolated distribution we obtain is itself 
induced by an acceptable window. 

Lemma 14. Suppose the algorithm A satisfies measure one correctness and termination and 
t := e~8^. Then, for any reachable configuration a not in Zq\JZ\, there exist sets R, Si, . . . , S n 



that can be applied to a such that the resulting reachable configuration falls outside Zq 1 U Z 

(t-i) 2 

with probability > 1 — 2e sn . 

Proof. Consider a reachable a in the complement of Zq U Z\. By definition of Zq, this means 
there is some choice of R, S such that applying R, S, . . . , S to a will avoid Zq~ 1 with probability 
> 1— r. Similarly, by definition of Z\, there is some choice R' , S' such that applying R' , S' , . . . , S' 
to a will avoid Z\~ x with probability > 1 — r. 

We assume without loss of generality that R' = {1,2, ... ,t'} for some t' < t. For each j 
from to n, we define the set Rj to be the union of R H {1, 2, . . . ,j} and R' D {j + 1, . . . , t'}. 
We observe that \Rj\ < t for each j. We also define Sf := S for i < j and Sj := S' for i > j. 
Then, for each j, we can apply Rj, S{, . . . , Si to a to produce a new reachable configuration. 
For each j, this induces a product distribution ttj on the set of reachable configurations. 

By construction, the distribution ttq places probability < r on Z^ 1 and the distribution ir n 
places probability < r on Z ~ l . The first j coordinates of 7Tj have the same distributions as in 



zk-l 



(t-ir 

7r n , will the remaining coordinates have the same distribution as in tto. We define r\ := e sn . 
We let j* denote the minimal value of j such that nj places probability < r/ on Zq" 1 . (Such a 
j* exists since j = n satisfies this condition.) If j* = 0, then ttq then places probability < rj on 
each of Zq -1 and Z^" 1 . Otherwise, we argue as follows. 

We use ¥ n .(A) for a set A to denote the probability that irj places on a set A. We claim 
that: 



B 



Z k Q-\\ 



z 



k-1 



(1) 



To see this, consider that the product distributions irj* and only differ in a single coor- 

dinate. Thus, if we sample a configuration according to and obtain a result in Zq -1 , we 

can resample the differing coordinate to match ttj* and we are guaranteed to obtain a result in 



B (z*- 1 , l). The inequality © follows. 



We observe that the set B{Z\~\t-\) is disjoint from the set B{Zq~ 1 , 1), since A(Zg~\ Z^ 1 ) > 



t. Hence, 



Combining ([T]) and ([2]), we see that 



B(z\- x ,t-\ 



> 



b ( zjr 1 , i 



(2) 



rk-l 



1 



B\Z\-\t-\ 



z 



k-1 



Applying Lemma [9] and recalling the definition of 77, we have 



Z 



k-1 



1 

< -e 



in 



We now have a product distribution ttj* induced by an acceptable window that places 



probability < r\ on each set Zq' 1 , Z\~ x , and hence P n Zq' 1 U Z\ 1 



< 2r/, as required. □ 
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4.3 Proof of Theorem [5] 



We now employ the notation and lemmas of the previous subsections to prove Theorem [5j We 

2 

define a := ^- and we define C sufficiently small such that 

1 (cn-l) 2 

Ce an < -eH^ (3) 
4 

holds for all positive integers n. For convenience of notation, we define E := Ce an . 

We consider the sets Zq ,Zf. By Lemma [T3"l we know that A (Zq,Z^ > t. We consider 
an initial configuration a in which all input bits are set to 0. Then, it must be the case that 
a ^ Zf . Otherwise, there would be a non-zero probability partial execution beginning with a 
and leading to a decision of 1, which contradicts measure one correctness. Similarly, an initial 
configuration 7 in which all input bits are set to 1 cannot belong to Zq. Hence, as we interpolate 
between a and 7, changing the input bit of one processors at a time, we must discover an initial 
configuration 5 such that 5 ^ Zq U Zf. We fix this setting of the inputs. 

Our strongly adaptive adversary is now defined as follows. Confronted with a partial execu- 
tion resulting in a configuration a, the adversary determines the maximum value of k < E such 
that a ^ ZqU Z\. If no such k exists, it continues arbitrarily within the constraint of producing 
acceptable windows. If such a k does exist, then it applies the sequence of sets guaranteed by 

mn • (cn-l) 2 

Lemma [L4J in order to yield a > 1 — 2e «« probability of reaching a new configuration at 
the end of the acceptable window that is not in Zq -1 U Z-^ . 

Since we begin with an initial configuration that is not in ZqUZ^, the probability that this 
strongly adaptive adversary will succeed in causing > E acceptable windows to occur before 
any decision is made is at least: 

(cn-l) 2 1 

1 - 2Ee ^~ > -, 

~ 2' 

recalling (J3j and the definition of E. This completes our proof of Theorem [5j 



5 Consequence for Resilience Against Crash Failures 

The techniques developed to prove the exponential lower bound in the previous section have 
implications beyond the strongly adaptive adversary. In fact, we can use the same techniques 
(with a few minor modifications) to prove a lower bound for more traditional asynchronous 
adversaries that applies to a large, natural class of algorithms. In particular, we consider an 
asynchronous adversary (with unbounded computational power and knowledge of all messages 
and internal states) that can cause up to t crash failures during an execution as well as controlling 
the message scheduling. The only constraint on message delivery is that all messages sent must 
eventually be delivered, if the recipient has not crashed. 

We now define the crucial properties of an algorithm that are needed to apply our lower 
bound techniques in this setting: 

Definition 15. We say an algorithm A is forgetful if each message sent by a processor depends 
only on its input bit as well as messages received and local randomness sampled since the previous 
sending event. 

Intuitively, this means that processors do not "remember" prior events that are not reflected 
by the most recently received messages. We define one more property of an algorithm that we 
will require in conjunction with forgetfulness: 

Definition 16. We say an algorithm A is fully communicative if whenever a processor receives 
the most recently sent messages from n—t processors, it sends a new message to all n processors. 
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These properties are both present in the algorithms in and seem natural in the context 

of crash failures, where one cannot wait for messages from t processors that may have crashed. 
We will prove that our exponential lower bound extends to forgetful, fully communicative al- 
gorithms against an adversary able to cause < t crash failures, making only minor semantic 
modifications to the proof in Section |H Intuitively, the combination of forgetfulness and full 
communication mimics the effect of the resetting failures we previously considered. Now proces- 
sors are retaining old information forever in their state, but they are basing current actions only 
on "recent" information, thereby proceeding as if they have forgotten the outdated portions of 
their internal state. 

In this context, we define a reachable configuration to be any configuration that occurs 
with non-zero probability with at most t crash failures (note that we have dropped the notion 
of acceptable windows). We analogously define measure one correctness and termination for 
algorithms by requiring that all reachable configurations display only valid combinations of 
input and output bits and that any infinite execution in which at most t crash failures occur 
and all other processors take infinitely many sending and receiving steps has probability zero. 

We will prove: 

Theorem 17. We set t = cn where c > is any fixed positive constant. Then there exist 
absolute positive constants C,a ( depending only one) such that, for any fully communicative and 
forgetful algorithm achieving measure one correctness and termination, there is an asynchronous 
adversary and a setting of the inputs bits such that with probability > \, the running time is 
> Ce an . 

5.1 Definitions and Lemmas 

We first adjust our definitions to obtain suitable sets Zq , Z\ for this setting. Since there are no 
longer any resets, we can assume without loss of generality that the local state of a processor 
includes a log of all messages the processor has received and sent throughout the execution so 
far. We will define all of our sets Z^,Z\ to be subsets of reachable configurations containing 
no crashed processors. We will rely on the fully communicative nature of the algorithm to 
additionally restrict to reachable configurations in which all processors are ready to send to all 
other processors. 

Given a reachable configuration a and sets Si, . . . , S n all of size > n — t, we say the adversary 
applies these sets to a to mean that the adversary executes the following sequence of steps. 
First, all processors taking sending steps. Then, each processor i receives the messages just 
sent to it from the processors in Si (in some fixed order). Note that when the algorithm is fully 
communicative, beginning from an initial configuration and repeatedly applying such n-tuples 
of sets will result in every processor sending to every other processor in each sending step. 

Now, analogously to the definitions in Section 14.21 we define: 

Definition 18. We let Zq denote the set of reachable configurations in S n such that at least 
one processor has written to its output bit. Similarly, we let Zf denote the set of reachable 
configurations in S n such that at least one processor has written 1 to its output bit. 

Definition 19. For k > 1, we let Zq denote the set of reachable configurations in S n where all 
processors are poised to send messages to all other processors and for any set \S\ > n — t, the 
adversary applying S,S, . . . , S to the configuration will result in a new configuration that belongs 
to Zq" 1 with probability > t. Similarly, we let Z\ denote the set of reachable configurations in 
S n such that, for any \S\ > n — t, the adversary applying S, S, . . . , S to the configuration will 
result in a new configuration that belongs to Z\~ x with probability > r. 
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We then have: 

Lemma 20. If a fully communicative and forgetful algorithm A satisfies measure one correct- 

t 2 

ness and termination and r > e~s^ ; then A [Zq,Z^\ > t for all non-negative integers k. 

Proof. We first establish the base case, i.e. that A (Z§, Zf) > t. This is similar to the proof of 
Lemma [TJJ 

We suppose there exist reachable configurations a, 7 G S n such that a G Zq, 7 G Z\, and 
A((J, 7) < t. Without loss of generality, we suppose that a and 7 only differ in the first t 
coordinates (i.e. only in the local states of processors 1 through t). We let S denote the set 
{t + l,...,n}. 

Consider a non-zero probability partial execution that results in configuration a. We can 
define another non-zero probability partial execution by executing most of the same steps, but 
crashing each of the first t processors before they send any messages that are not sent in a partial 
execution resulting in 7. In other words, we reach a new configuration 5 that agrees with a, 7 
in the final n — t coordinates, and the steps taken by the first t processors in 5 are precisely the 
common steps taken by these processors in both o~,j: at the point where the actions of these 
processors diverge in a and 7, the processors are crashed. 

Now 6 is reachable, and the adversary can continue a partial execution from 5 by continually 
executing sending and receiving steps among the final n — t processors. Since the algorithm 
A satisfies measure one termination, if the adversary keeps extending this execution, with 
probability one a decision must eventually be reached. Let's suppose that this decision is 1 with 
non-zero probability. Then the same extension can be applied to a partial execution reaching 
a and this will yield conflicting decisions with non-zero probability, contradicting measure one 
correctness. Similarly, if the decision reached from extending 5 is with probability 1, then a 
partial execution reaching 7 can be extended to yield conflicting decisions. We may conclude 
that A(Z$,Z$) > t. 

We now proceed by induction on k (similarly to the proof of Lemma [T3|) . We assume 
the result holds for k — 1 > 0, and we suppose it is false for k. Then there exist reachable 
configurations a, 7 G S n such that a G Zq, 7 G Z\, and A(<r, 7) < t. Without loss of generality, 
we suppose that a and 7 only differ in the first t coordinates. We let S denote the set {t + l,t + 
2, . . . , n}. By definition of Zq, if the adversary applies S, . . . , S to a, this will with probability 
> r result in a new configuration belonging to Z ~ l . If the adversary applies S, . . . , S to 7, this 
will with probability > r result in a new configuration belonging to Z^ 1 . 

We first consider the case where no output bits have yet been written in a or 7. By the 
forgetful and fully communicative properties of the algorithm, the distributions of the configu- 
rations resulting from applying S, . . . , S to a and to 7 only differ in portions of the local state 
that can no longer affect behavior of the processors going forward. This is because the new 
messages to be sent by all n processors will only depend on the input bits and the newly received 
n — t messages, not the prior portions of the state that differed between 7 and a. Hence, the 
product distribution induced by applying S, . . . , S to 7 places weight > r of each of two sets, 
Zq~ 1 and Z^ 1 , that are separated by a Hamming distance > t. Applying Lemma El we thus 
have that 

,2 .2 
o t t 

r < e 4 « 44> r < e sn . 

This contradicts our stipulation on the value of r. 

In the case that an output bit has already been written as in a, say, then we reach a 
contradiction by repeatedly applying S, . . . , S to a. Since 7 G Z\, there is a nonzero probability 
that this results in a decision of 1 by processors outside of the first t, since these are unaffected 
by the first t processor states when these processors are no longer heard. □ 
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Lemma 21. Suppose a fully communicative and forgetful algorithm A satisfies measure one 

-in- 
correctness and termination and r := e 8n . Then, for any reachable configuration a not in 

2e- 



Zn U Zt, there exist sets S\, . . . ,S n that can be applied to a such that the resulting reachable 



configuration falls outside Zq 1 U Z* 1 with probability > 1 



rk-1 



8n 



Proof. This is essentially the same as the proof of Lemma [LA but we restate the argument 
for completeness. Consider a reachable a in the complement of Zq U Z\. By definition of Zq, 
this means there is some choice of S such that applying S, . . . , S to a will avoid Zq -1 with 
probability > 1 — r. Similarly, by definition of Z^ > there is some choice S' such that applying 
S', . . . j S' to a will avoid Z^ -1 with probability > 1 — r. 

We define Sf := S for i < j and 5/ := S' for i > j. Then, for each j, we can apply S{ , . . . , Si 
to o" to produce a new reachable configuration. For each j, this induces a product distribution 
7Tj on the set of reachable configurations. 

By construction, the distribution ttq places probability < r on Zf -1 and the distribution 7r n 
places probability < r on Zq -1 . The first j coordinates of iij have the same distributions as in 



7T n , while the remaining coordinates have the same distribution as m ttq. We dehne r] := e sn . 
We let j* denote the minimal value of j such that ttj places probability < r\ on Zq -1 . (Such a 
j* exists since j = n satisfies this condition.) If j* = 0, then ttq then places probability < rj on 
each of Zq -1 and Z\~ x . Otherwise, we argue as follows. 

We use F n .(A) for a set A to denote the probability that ttj places on a set A. We claim 



that: 



B Z, 



7 k-l 



> 



7 k-l 



(4) 



To see this, consider that the product distributions 7r 3 * and 7r ? *_i only differ in a single coor- 



dinate. Thus, if we sample a configuration according to vrj*_i and obtain a result in Zq 1 , we 
can resample the differing coordinate to match 7r 3 * and we are guaranteed to obtain a result in 



B (Zq~ 1 , l). The inequality © follows. 



We observe that the set B(Zf~\ t-l) is disjoint from the set B{Z^~ 1 , 1), since A(Zq~\ Zf _1 ) > 
t. Hence, 



B(Z 



yk-l 



t - 1 



> 



(5) 



Combining ^ and ([5]), we see that 



7 fc-i 



Applying Lemma [9l we have 



B\Z\-\t-\ 



Z\ 



fe-i 



1 (t-i) 2 

< — g 4n 

7? 



z 



fc-l 



We now have a product distribution 7r 3 * induced by applying a sequence of sets of size > n — t 



that places probability < r/ on each set Zq \z\ 1 , and hence P„- , 
required. 



Zq*- 1 U Zf _1 



< 277, as 
□ 



5.2 Proof of Theorem [T7] 

We define a, C, E 1 as in Section POl and consider the sets Zq , Zf' as defined above. There must 
be an initial configuration <5 such that 5 ^ Z^ U Zf '. We fix this setting of the inputs. 

Beginning with <5, the adversary proceeds as follows. It first applies the sequence of sets 

. (cn-l) 2 

guaranteed by Lemma [21] in order to yield a > 1 — 2e »n probability of reaching a new 
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configuration that is not in Zq 1 U Zf 1 . If it succeeds, it can then apply a new sequence of 

en-l) 2 

sets (again furnished by Lemma 121]) in order to yield a > 1 — 2e sn probability of reaching 
a new configuration that is not in Zq ~ 2 U Zf~ 2 , and so on. 

The probability that this adversary will succeed in causing > E such iterations to occur 
before any decision is made is at least: 

(cn-1) 2 1 

1 - 2Ee ~ > -, 

recalling ([3]) and the definition of E. By the fully communicative property, we know that in 
each iteration, every processor will send a message to every other processor. This guarantees a 
message chain of length E. This completes our proof of Theorem [T71 
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